The Test Bus Imperative: Architectures That Support Automated Acceptance Testing

A gile methods—more specifically, test-driven development practices 1 —have begun to raise the software industry's awareness of automated acceptance testing. We now recognize that armies of manual testers slogging through reams of tedious, manually executed test scripts, under schedule pressure at the end of a release, don't ensure quality. Many tools can be purchased to help testers write automatic scripts for testing the system through its user interface. Such tools act like robots programmed to behave like testers—they push buttons, select menu items, check boxes, enter data into fields, and inspect the screens. Unfortunately, testing through the UI is slow, opaque, and dangerous. One of my clients had to run over 10,000 acceptance tests, and running them through the UI took him several days using dozens of machines. Furthermore, the tests were written in a code-like language that made them difficult to understand. Over time, he lost track of what each test was trying to verify: all he knew was that all 10,000 tests had to pass. Even tiny changes to the UI caused dozens, if not hundreds, of tests to fail or become inoperable. In the end, he found it impossible to upgrade and improve his outmoded UI. Over a century ago, the telephone company solved a similar problem by designing telephone switches with built-in test access. Such access, called a test bus, eventually let the telephone company automatically test every phone line at night and fix failures and degradations long before subscribers learned about them. Software producers have started adopting a similar strategy, architecting their software systems with built-in test buses. They also use tools such as Fit and FitNesse (www.fitnesse. org) to specify their tests in a simple-to-understand specification language that business people can read. 2 However, setting up automated acceptance testing requires at least three major architectural separations. In a software system, a test bus is a set of APIs that provides convenient access to unit and acceptance tests. These tests specify system behavior—unit tests specify module behavior and acceptance tests specify feature behavior. A test bus's existence in a system means that the appropriate structures exist in the application to let tests access the modules and subsystems whose behavior and structure they're specifying. For example, writing tests that bypass the UI and exercise the underlying business rules requires an API that both the UI and tests can use to invoke those business rules. Moreover, that …